Increasingly, digital data is embedded in the audio and visual material that we encounter every day. An example are UPC codes that appear as familiar stripes on supermarket products and elsewhere. These are read and decoded by a special scanner, for example, at the checkout counter of the supermarket, or by handheld readers, and the data encoded therein is interpreted and used for many purposes, “QR” code's are another example, these are two-dimensional, printed or displayed codes that are designed to be interpreted by a simple camera. QR codes have become popular for many applications and are widespread. However, they are demarcated codes and most applications require that the detector be correctly aimed and centered on the code.
As more and more public and public space displays become electronic, there is a need for a code that is optimized for these displays. In addition, because these displays can change dynamically, such a code would optimally be able to be changed almost instantly and can also display an ever-changing, or a streamed set of information.
This invention relates to the encoding and decoding of streamed data and more particularly to a system and method for the display of pictures that include codes invisible to a user but detectable by a machine such as a camera along with associated computing capability. In addition, the invention describes the systems and methods for detecting and decoding those embedded codes.
The codes, called “VR codes,” are signals preferably but not necessarily invisibly hidden in and around pictures in a video display such as a television screen or computer monitor that convey a stream of information to a receiver. An example receiver is a camera in a cell phone. VR codes are similar to QR codes in that they are digital tags. VR codes are an improvement over QR codes in that VR codes are a dynamic stream and can be embedded in a picture in a way that is not visible to a user. Such codes are thus an example of using displays simultaneously to carry information useful to a person and to a machine. VR codes can be made visible if so desired.
There has been related prior art work in two areas. In a first area, data is embedded in a picture to mark the picture in some way. This technique is generally called steganography. This data is intended to be digitally decodable by processing a digital signal, not from the display of the picture itself. There is also well-known work called “digital watermarking” where a pattern is embedded in a sound or picture to identify it. In other prior art work, there are visible patterns on a screen at which one can aim a simple detector in order to detect the presence of a pattern. These techniques have been used for video games. For example, a detector can be a photodiode with a lens built into a toy gun, and when one aims at a visible target, the detector notes a hit. The VR codes disclosed herein are novel in that they are optimally invisible codes that are detectable by a normal camera.
What is needed is a code that can be optimized for dynamic displays detected by simple, existing cameras such as those people have in mobile and portable devices, and that can carry information that does not intrude on the enjoyment or message of the pictures, and is detectable by the cameras we already have in mobile phones, tablets, computers and portable cameras.
It is an intention of this invention to create a code that can be embedded in a display that is invisible or minimally noticeable by people but is readily detectable and decodable by the camera and processing in a mobile device.
It is a further goal of this invention to make a code that changes with time so that a large amount of data can be transferred to the mobile device and so that codes can be readily changed.
It is yet a further goal of this invention to allow the codes to be represented in the periphery of the region of the display that is occupied with the image as well as be embedded within the image itself. The codes can be made visible so that a user can know they are there, or they cart be made invisible so that they add no readily perceptible alteration to the displayed image or image sequence.
It is yet a further goal of this invention to make the codes use an area of the screen so that the user need not aim the camera or detector, at a specific region of the screen or center it deliberately in a particular area of the frame that the camera captures.
It is a further intention of this invention to make codes that occupy a large area in and surrounding the active area of a display so that the geometric distortion of the coded display on the image plane of the camera can be used to identify the position of the viewer. I.e., the manner by which the code is detected can also be used to identify whether the user is directly in front of the display or is off axis to the left, right, high or low. The ability to use a large screen area as opposed to a small, demarcated region allows this determination to be more accurate than with localize codes such as QR codes.